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OPTIMAL CONTROL OF A LARGE DAM 


VYACHESLAV M. ABRAMOV 


Abstract. A large dam model is an object of study of this paper. 
The parameters L lower and L upper are its lower and upper levels, L = 
j^upper _ flower j s i ar g ej anc [ jf a current level of water is between these 
bounds, then the dam is assumed to be in normal state. Passage one 
or other bound leads to damage. Let Ji (J 2 ) denote the damage cost 
of crossing the lower (upper) level. It is assumed that input stream of 
water is described by a Poisson process, while the output stream is state- 
dependent (the exact formulation of the problem is given in the paper). 
Let Lt denote the dam level at time f, and let p\ = limt^oo P{G = 
P 2 = Hindoo P {Lt > L upper ] exist. The long-run average cost 
J = pi Ji + P2J2 is a performance measure. The aim of the paper is to 
choose the parameter of output stream (exactly specified in the paper) 
minimizing J. 


1. Introduction 

A large dam model is an object of study of this paper. The parameters 
jjower anc j j^upper are j ower anc j U pper levels of the dam, and if a current 
level of water is between these bounds, then the dam is assumed to be in 
normal state. The reason that a dam is called large is that the difference 
L = L upper — l) ower is large. This assumption enables us to use asymptotic 
analysis as L —> 00 and obtain much simpler representations for the desired 
characteristics of the model. In turn these representations are then easily 
used to solve the appropriate control problems formulated below. 

In the literature, the dam, storage and production models are associated 
with state-dependent queueing systems (e.g. [I], [ 2 ], 0, 0, 0, pZO, HU, 
m, pi , m and others). The model of the present paper is the follow¬ 
ing. We assume that units of water, arriving to a dam, are registered by 
counter at random instants t,\ , t^,... , and interarrival times r n = t n+ \ — t n 
are mutually independent exponentially distributed random variables with 
parameter A. Outflow of water is state-dependent as follows. If the level of 
water is between L lower and L upper , then an interval between unit departures 
has the probability distribution B\(x). If the level of water increases above 
the level L upper , then the probability distribution of an interval between unit 
departures is B2(x). It is also assumed that if the level of water is exactly 
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of the level L lower , then the departure process of water is frozen, and it is 
resumed again as soon as the level of water increases the value L lower . It 
is worth noting that the policies at which the service rate is changed in de¬ 
pendence of a dam level are of notable attention in the literature (see [1], 
0 , m, 0 , m and other papers). However all of them discuss performance 
measures associated with an appropriate upper level of water in a dam and, 
to our best knowledge, the known results on performance analysis of river 
flows structured by lower and upper levels are analytically difficult and hard 
for real applications even for simple models (e.g. see review paper HU)- 
Furthermore, in the most studies the explicit representations are in terms 
of the Laplace-Stieltjes transforms of the initial distributions, and there is 
no paper providing asymptotic analysis of large dams. 

In terms of queueing theory the problem considered in the paper can be re¬ 
formulated as follows. Consider single-server queueing system where arrival 
flow of customers is Poisson with rate A, and a service time of a customer de¬ 
pends upon queue-length as follows. If at the moment of service begun, the 
number of customers in the system is not greater than L , then the service 
time of this customer has the probability distribution B\(x). Otherwise, if 
there are more than L customers in the system at the moment of service 
begun, then the probability distribution function of the service time of this 
customer is B 2 (x). The analytical results for this queueing system are known 
(e.g. Abramov 0)- Notice that the lower level of dam L lower is equated 
with an empty queueing system. Then the dam specification of the prob¬ 
lem is characterized by performance criteria, which in terms of the queueing 
formulation looks as follows. Let qt denote the queue-length in time t. The 
problem is to choose an output parameter of system minimizing the func¬ 
tional J(L) = pi(L)J\(L) + p 2 (L)J 2 (L), where pi(L) = lim i ^ 00 P{% = 0}, 
p 2 (L) = lim^ooPl^ > L}, and J\(L) and J 2 (L) are the corresponding 
damage costs proportional to L. To be precise assume that J\(L) = j\L 
and J 2 {L) = j 2 L, where j\ and j 2 are positive constants. Assuming that 
L -> oo we shall often write p\ and p 2 (without argument L) rather than 
Pi(L) and p 2 {L). The argument L will be often omitted in other functions. 
We shall feel free to write J, J± and J 2 without the argument L. 

To specify the problem more correctly we assume that the input pa¬ 
rameter A, and probability distribution function B- 2 (x ) are given, while 
B\(x) = B\(x,C) is a family of probability distributions depending on 
parameter C > 0, which in turn is closely related with the expectation 
f 0 xdBi(x). Then the output rate, associated with this probability distri¬ 
bution Bi(x), can be changed so that the minimum value of the functional 
is associated with the choice of this parameter C, resulting in the choice of 
the function B\(x, C). The correctness of such formulation and more con¬ 
crete clarification of parameter C will be explained in the sequel (see the 
formulations of Theorem 14.11 14.21 14.31 14.41) . It is interesting to note, that 
the solution of the above control problem is asymptotically independent of 
the explicit form of probability distribution functions B\(x) and B 2 (x), and 
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only depends on the expectations f^° xdB 2 (x) < j and / 0 °° xdB\{x) as well 
as x 2 dB 1 (x). The details of this dependence will be explained later. 

We use the notation bi = J 0 °° xdBt(x), pi = A (i = 1,2) and as¬ 
sume that p 2 < 1. This assumption is a standard condition of stationar- 
ity, ergodicity of the queue-length process qt and existence of the limits p\ 
and p 2 (independent of any initial state of the process). In additional to 
this assumption we shall also assume the existence of the third moment: 
Pi,k = A fc f 0 °° x k dB\{x) < oo, k = 2, 3. The existence of the second moment 
is used in Theorem 13.21 Then the existence of the third moment for all the 
specified family of distributions Bi(x,C ) is required in Theorem 14.11 14.21 

rasai 

The special features of the present paper are as follows. 

• We solve the control problem where the performance criteria takes into 
account passage of the upper and lower bounds. The formulation of the 
problem is not traditional but realistic. Other similar control problems 
arising in practice can be flexibly solved by adapting the method of this 
paper. 

• The presentation of the results are clear and available for real applica¬ 
tion. 

• The mathematical methods of the paper are not traditional but clear 
and easily understandable. 

The paper is structured as follows. In Section 2 we discuss the state- 
dependent queue-length process and derive representation for the probabil¬ 
ities pi and p 2 ■ Section 3 contains the results on asymptotic analysis of 
probabilities p± and p 2 , and the main result of this asymptotic behaviour is 
given by Theorem [321 In Section 4 some additional theorems on asymptotic 
behaviour of p± and p 2 are proved, which are then used to solve the control 
problem. The main result of this paper, the solution of control problem is 
formulated in Section 5. Concluding remark are given in Section 6. 

2. The state-dependent queue and its characteristics in a busy 

period 

In this section we discuss the main characteristics of the state-dependent 
queueing system described in the introduction. Let Tl, II and vl denote 
correspondingly a busy period, an idle period and the number of served 
customers during a busy period. Let T^\ denote the total time during 

a busy period when correspondingly 0 < qt < L and qt > L, and let , 
( 2 ) 

v y L ; denote correspondingly the total numbers of served customers during 
a busy period when correspondingly 0 < qt < L and qt > L. We have the 
following two obvious equations: 

(2.1) e t l = etJ 1} + E r£ 2) , 

(2.2) 


bu l = e4 1} + e4 2) . 
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According to the Wald’s equation, 

(2.3) ETP = 6iE 4 1} , 
and 

(2.4) ET^ ) = b 2 E^ ) . 

Next, the number of arrivals during a busy circle coincides with the total 
number of served customers during a busy period. Hence, applying the 
Wald’s equation again and taking into account ([2.1I) - (12.4D . we obtain 

XETl + XEI = XETl + 1 

= AE r£ 1} + AE r£ 2) + 1 
(2 ' 5) = piEvP + p 2 Evf } + 1 

= e4 1} +e4 2) . 


From (12.51) we have the equation 


( 2 . 6 ) 


e 4 2) = 

L 1 - P2 1 - P2 


^e44 


expressing Ev^ via e 4^- For example, if p\ = 1, then e 4^ = 4 f° r 
any L. 

The similar equation holds also for ET^ . Namely, from (12. 4|) and (12.61) 
we obtain 


(2.7) 



Pi _ 12(1 ~ Pi) Er (i) 
A(1 — P 2 ) A(1 — P 2 ) L 


Equations (|2.61) and (|2.71) enables us to obtain the stationary probabilities 
pi and p 2 - Applying the renewal reward theorem (e.g. Ross m p. 78) and 
consequently (12.5j) and (12.61) . for p\ we obtain: 


( 2 . 8 ) 


El 

e 4 1} + ETf } + El 
1 

e4 1} + e4 2) 

1 - P'2 


1 + (pi - P2 )e4 1} 
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Analogously, 


(2.9) 


e r£ 2) 

E T ( r 1] + E t\ 2) + El 

Eu { ^ + E4 2) 

Pi + P2(pi - 1)e4 1} 

1 + (pi - P2)e4 1} 


3. Asymptotic analysis of pi and p 2 as L increases to infinity 


By sample path analysis and the property of the lack of memory of ex¬ 
ponential distribution it follows that the random variable coincides in 
distribution with the number of served customers during a busy period of 
the M/GI/l/L queueing system (the parameter L denotes the number of 
customers in the system excluding the customer in the server). Specifically, 
we use the fact that during a busy period the number of times service begun 
when the number of customers in the system does not exceed L, coincides 
with the number of arrivals when the number of customers in the system 
does not exceed L + 1. We also use the fact that the residual interarrival 
time after a service completion has exponential distribution with parameter 
A. 

Therefore the known results of the M/GI/l/L queueing system can be 
used. 

It is known (e.g. 0,0) that EiJj ] * is determined by the convolution type 
recurrence relation 


Ez4 1} = 


3=0 ° 


,-Ax (As)- 7 ' 


j- 


d£i(x), E^ 1} = 1 


where Eu r P denotes the expectation of the number of served customers 
during a busy period of M/GI/l/n queue (n = 1,2,...). 

The probabilities Pi(L) and P 2 (L) are expressed explicitly via Ev^, and 
their asymptotic behavior as L —► oo can be obtained from the following 
known results. 

Let Qo > 0 be an arbitrary real number, and for n > 0 


Qn — ^ ( T'jQn—j+l , 

3=0 


where ro > 0, rj > 0, and tq + r\ + ... = 1. Let r(z) = YlpLo \ z \ — 1 
be a generating function, and let = lim z n where A m \z) is the 

mth derivative of r(z). 
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(3.1) 


Notice, that the sequence {Q n } is an increasing sequence, and 

Qor(z) 


/*N Qr 


z = 


n =0 


z — r{z) 

(see m, p] . Sect. 25 and [20]). 

Lemma 3.1. (Takacs [20], p. 22-23). If 71 < 1, then 

Qo 


lim Q n = 


// 7 i = l and 72 < 00 , then 


lim 

n—>oc ji 


1-71 
Qn 2Qo 


72 


7/7i > 1, t/ien 


lim 

n—>oo 


Qn 


Qo 


<7 n (1 — r'(a)) 1 — 71 ’ 

where a is the least in absolute value root of functional equation z = r(z). 

From this lemma we have the asymptotic results for the probabilities p\ 
and P 2 ■ 

For 5R(s) > 0 denote by 7?i(s) the Laplace-Stieltjes transform of B\(x). 
We have the following theorem. 


Theorem 3.2. If p\ < 1, then 

(3.2) 

(3.3) 


lim pi (L) = 1-pi, 

L—>00 

lim po(L) = 0 . 

L —>00 


If pi = 1, then 

(3.4) 

(3.5) 

If pi > 1, then 

(3.6) lim 

L —>00 pi — p 2 

where p is the least in absolute value root of the functional equation z = 
Bi(\ — A z), and 

lim P 2 {L) = P2<yPl -—. 

L — >00 


lim Lpi(L) = 

L —>00 Z 

lim Lp 2 {L) = 

L^oo 1 — P2 2 


Pi{L) (l-p 2 )[l + XB[(X-\p)} 


(3.7) 


Pi - P2 
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Proof. The proof of this theorem follows by application of Lemma 13.11 
Straightforward application of the aforementioned lemma to the recurrence 
relation for Ez^ yields the following. 

If pi < 1, then 

(3.8) lim Ez^ =-. 

L—hx> 1 — pi 


If pi = 1, then 


(3.9) 


lim 

L —>oo 



2 

Pi,2 


If pi > 1, then 


(3.10) 


lim 

L —>oo 



1 

ip L (l + XB[(X - \z)) 


1 

1 -Pi 


Substituting (13.8H - d3.10l) for the limits in (12.81) and (12.91) correspondingly to 
the cases in the formulation of the theorem finishes the proof. □ 


4. Further asymptotic analysis of pi and p 2 

Let us first discuss the statements of Theorem l3.21 Under the assumption 
pi < 1 we have (|3.2I) and ([3.31) . The probability p\ is positive in limit while 
the probability p 2 vanishes. Under the assumption pi > 1 we have (|3.6I) and 
(HZ}. According to these relations the probability pi vanishes while p2 is 
positive in limit. This means that if both Ji and J 2 are large positive values 
proportional to L , then the functional J will take the value proportional to 
a large parameter L too. Specifically, in the case pi < 1 for this value we 
have J k, (1 — pi)J\, and in the case pi > 1 we have J ~ £^£Lz2) j 2 . 

In the case pi = 1 both pi and P 2 vanish with the rate L , and therefore 
J converges to the limit as L —> 00 . Thus, the case pi = 1 is a possible 
solution of the control problem, while the cases pi < 1 and pi > 1 are 
irrelevant. Specifically, for J = J(L) we obtain the following: 


(4.1) 


lim J(L ) - + j 2 

L—>00 z 


P2 

1 - P2 


Pi,2 

~Y 


In order to find now the optimal solution consider the following two cases: 
(i) pi = 1 + 5 and (ii) pi = 1 — 5, where in these both cases 5 —> 0 as 
L — > 00 . 

In case (i) we have the following two theorems. 


Theorem 4.1. Assume that pi = 1 + S, 5 > 0, and L5 ^ C > 0, as 5 —> 0 
and L —► 00 . Assume that pi$ = pi,3(<5) is a bounded function of parameter 
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5, for all 0 < 5 < 1 and there exists p\ i2 = lim^o Pi, 2 (d)- Then, 


(4.2) 

Pi 

(4.3) 

P2 


+ o(d), 


fPC/pi^ _ 1 

dp‘2 efC/pi,2 

(1 - p 2 ){e 2C /P 1 ’ 2 - 1 ) 


+ o(5). 


Proof. The proof of this theorem is similar to that of Theorem 3.4 of [5] 
and Theorem 4.4 of |6j. Under the conditions of the theorem the following 
expansion was shown in Subhankulov |T8], p. 326: 

(4.4) tp = 1- J^- + 0(5 2 ). 

Pi ,2 

Then, by virtue of (14.41) after some algebra we have: 

(4.5) 1 + \<p) = 5 + 0(5 2 ). 


Substituting (14.41) and (14.51) for (13.101) we obtain: 


(4.6) 


Evi 1} = 


3 2C/pi,2 _ ^ 


+ 0 ( 1 ). 


From (14.61) and (12.81) and (12.91) we finally obtain the statement of the theo¬ 
rem. □ 


Theorem 4.2. Under the conditions of Theorem \f.l\ assume that 0 = 0. 
Then, 


(4.7) 

(4.8) 


hm Lpi(L) = 

L—>00 Z 


lim Lp 2 (L) = 

L—> 00 


P2 


1 - P2 


Pi,2 
2 


Proof. The statement of the theorem follows by expanding the main terms 
of asymptotic relations of (14.21) and (14.31) for small C. □ 


Notice, that (14.71) and (14.8|) coincide with (13.41) and (13.51) correspondingly. 
In case (ii) we have the following. 

Theorem 4.3. Assume that p\ = 1 — 6, 5 > 0, and L5 — > C > 0, as 6 —> 0 
and L —> 00 . Assume that p\^ = P\;.i{d) is a bounded function of parameter 
5, for all 0 < 5 < 1 and there exists p\p = lim^ 0 Pi,2(5). Then, 

(4.9) pi = 5e ?1 ’ 2/2C + o(5), 

(4.10) P 2 = 5 • (e^ 2C - l) + o(5). 

I- p 2 \ > 

Proof. From (13.11) we have 

Ve = , 

h ; Bi(\ — \ z ) — z 
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The sequence {Ei^} is an increasing sequence, and in the case p± = 1 from 
the Tauberian theorem of Hardy-Littlewood (e. g . m, ns], m, Q2i> m) 
we obtain: 


Et' 


(i) 


lim 

L — >oo L 


= lim(l- z ) 2 T l(A ~ Az) . 
*n B 1 (X-Xz)-z 


(It is not difficult to check that then (|3.9I) follows.) Then in the case where 
pi = 1 — 5, and Lb —> C as L —> oo, according to the same Tauberian 
theorem of Hardy and Littlewood, asymptotic behaviour of Ezzj 1 ) can be 
found from the asymptotic expansion 


(4.11) 


(!-*)■ 


Bi (A - A z) 


B\ (A — A z) — z 

as z | 1. 

By the Taylor expansion of the denominator of (|4.11jl we obtain: 
1-z 1-z 


B i(A Xz ) z 1 - z - P i(l - z ) + ^(1 - z ) 2 + 0((1 - z) 3 ) 

1 


(4.12) 


b+ p -f(l-z) + 0[(l-zY] 


b[l + P -^{l-z) + 0[(l-zY] 

1 


[1 + 0 (1)]. 


6 exp 


P 1,2 

1 lb 


(1 - z) 


Therefore, assuming that z = 


the asymptotic behaviour of Ez^) ; as L —>• oo. We have: 


L—l 

L 

( 1 ) 


1 as L —>• oo, from (14.121) we obtain 


(4.13) 


,(i) 


1 


T7,a_ 

L ~ be^/ 2C 


[l + o(l)]. 


Now, substituting (14.131) for 
of the theorem. 


and (12.9p we obtain the desired statements 

□ 


Theorem 4.4. Under the conditions of Theorem 4-3 assume that (7 = 0. 
Then we obtain (H2D and 

Proof. The statement of the theorem follows by expanding the main terms 
of asymptotic relations of (14.91) and (I4.10P for small C. □ 


5. Solution of the control problem 

In this section we formulate the theorem characterizing the solution of 
control problem. 
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For J = J(L) we have the following limiting relation 


(5.1) 


lim J(L) 

L —>oc 


lim [pi(L)Ji(L) + p 2 (L)J 2 (L)\ 

L—>oc 

ji lim Lpi(L) + j 2 lim Lp 2 (L). 

L—>oo L —»oo 


Substituting (14.11) and (14.21) for the right-hand side of (|5.1I) and taking into 
account that L5 —* C, we obtain: 


jupper = Um J(L) 
L —>00 


(5.2) 


1 


Ji 


= C 


q2C/p\,2 _ l 

1 


+ 42 


/?2e 


2C/pi t 2 


(1 - p 2 )(e 2C /P 1 ’ 2 - 1) 
2C/pi,2 


lim L5 

L —>00 


3i- 


2 C / Pi , 2 _ i +J2 (1 _ P 2 )(e 2C /p 1,2 - l) 


/>2e 


Substituting (14.91) and (|4.10p for the right-hand side of (15.11) and taking into 
account that L5 —* C, we in turn obtain: 


(5.3) 


J 


lower 


= c 


he^ 2C +h-^(e^ c -l) 

1 - P 2 V / 


Let us now study the functionals J u PP er and J lower given by (15.21) and (15.31) . 
Observing (15.21) . notice that there contain the constants j \, j 2 and p 2 in 
m- Let us assume that these constants are given such that 


(5.4) 
Then C 


(5.5) 


Ji =32 


P 2 


1 - P2 

0 is the point of min of the functional J u PP er . Indeed, in this case 


jupper = nC 

= 3iC 

= 3iC 


1 


a 2C'/pi l 2 


+ 


e 2C/pi,2 _ \ q2C/pi,2 _ ]_ 


1 (e 2C / Pl ' 2 - 1) + 1 


e 2C/pi,2 _ l 
2 


+ 


3 2C/pi,2 _ l 


1 + 


j2C/pi,2 _ \ 


Therefore in point C = 0 we have lirnc^o J upper = jipip, and in the right 
side of the point C = 0 the function J u PP er is increasing in C. Hence (15.41) 
is the condition for C = 0. 

Next, 


(5.6) 


C 


3 2C/pi,2 _ \ 


J C 


9C 2 

e 2C/pi, 2 _i_ . e 2C/pi, 2 

Pl ,2 

(, e 2C/pi, 2 _ 1)2 
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and 


(5.7) 


’ Ce 2C / p !' 2 

/ 

C 

e 2cypi,2 _ i 

c 

e 2C/pi,2 _ l 


e 2C/pi,2 



Ce 2C / p1 ’ 2 

e 2C/pi,2 — l 


qZC/pi^ 


Clearly that (EJD is not smaller that m, and they are equal when 
C = 0. 

Therefore, if the right-hand side of (15.411 is greater than that left-hand 
side of (15.411 . then C = 0 remains to be the value minimizing the functional 
jupper rp^ e g j m jj ar resu it holds for functional J lower given in (15.31) . Specifi¬ 
cally, if the right-hand side of (15.41) is not greater than the left-hand side of 
(15.41) . then C = 0 remains to be the value minimizing the functional J lower . 

Thus, the solution of control problem is given by the following theorem. 


Theorem 5.1. If the parameters A and p 2 are given, then the optimal so¬ 
lution of the control problem is the following. 

•If 


Ji = 


P2 


-3 2 , 


1 — P2" 

then the optimal solution of the control problem is achieved for p\ = 1. 

•If 

■ ^ P2 . 

31 > z - 32, 

1 — P2 

then the optimal solution of the control problem is a minimization of the 
functional J u PP er . The optimal solution is achieved for p\ = l + < 5 , 5(L) is a 
small positive parameter, and L5(L) —► C. C is the nonnegative parameter 
minimizing m- 

• u 


31 < 


P2 


-32, 


1 - P2' 

then the optimal solution of the control problem is a minimization of the 
functional J lower . The optimal solution is achieved for p\ = 1 — 6, 5(L ) is a 
small positive parameter, and L5(L) —► C. C is the nonnegative parameter 
minimizing (USD- 


6. Concluding remarks 

In this paper we posed and solved a control problem for a large dam. The 
main specification of the problem is that the performance criteria takes into 
account passage the lower and upper bounds. The solution of the control 
problem is asymptotically independent of the explicit form of probability 
distribution functions B\(x) and B 2 {x), and under the assumption that the 
parameters A and p 2 are given, in dependence of a performance criteria 
the parameter p± must have one of the forms: p\ = 1, p\ = 1 + 5(L), or 
pi = 1 — S(L) where 5{L) > 0, and as L —> oo, 6(L) vanishes and L6(L) —> C. 
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